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Abstract 

We study connections between Natural Proofs, derandomization, and the problem of proving "weak" 
circuit lower bounds such as NEXP (£ TC°, which are still wide open. 

Natural Proofs have three properties: they are constructive (an efficient algorithm A is embedded in 
them), have largeness (A accepts a large fraction of strings), and are useful {A rejects all strings which 
are truth tables of small circuits). Strong circuit lower bounds that are "naturalizing" would contradict 
present cryptographic understanding, yet the vast majority of known circuit lower bound proofs are nat- 
uralizing. So it is imperative to understand how to pursue un-Natural Proofs. Some heuristic arguments 
say constructivity should be circumventable. Largeness is inherent in many proof techniques, and it is 
probably our presently weak techniques that yield constructivity. We prove: 

• Constructivity is unavoidable, even for NEXP lower bounds. Informally, we prove for all "typical" 
non-uniform circuit classes C, N EXP (£_ C if and only if there is a polynomial-time algorithm distin- 
guishing some function from all functions computable by C-circuits. Hence NEXP <f_ C is equivalent to 
exhibiting a constructive property useful against C. 

• There are no P-natural properties useful against C if and only if randomized exponential time can be 
"derandomized" using truth tables of circuits from C as random seeds. Therefore the task of proving 
there are no P-natural properties is inherently a derandomization problem, weaker than but implied by 
the existence of strong pseudorandom functions. 

These characterizations are applied to yield several new results. The two main applications are that 
NEXP n coNEXP does not have n logn size ACC circuits, and a mild derandomization result for RP. 



I Introduction 

The Natural Proofs barrier of Razborov and Rudich [24] argues that 

(a) almost all known proofs of non-uniform circuit lower bounds entail efficient algorithms that can dis- 
tinguish many "hard" functions from all "easy" functions (those computable with small circuits), and 

(b) any efficient algorithm of this kind would break cryptographic primitives implemented with small 
circuits (which are believed to exist). 

(A formal definition is in Section 2.) Natural Proofs are self-defeating: in the course of proving a weak 
lower bound, they provide efficient algorithms that refute stronger lower bounds that we believe to also 
hold. The moral is that, in order to prove stronger circuit lower bounds, one must avoid the techniques used 
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in proofs that entail such efficient algorithms. The argument applies even to low-level complexity classes 
such as TC° [22, 19], so any major progress in the future depends on proving un-Natural lower bounds. 

How should we proceed? Should we look for proofs yielding only inefficient algorithms, avoiding 
"constructivity"? Or should we look for algorithms which cannot distinguish many hard functions from 
all easy ones, avoiding "largeness"? 1 (Note there is a third criterion, "usefulness", requiring that the proof 
distinguishes a target function / from the circuit class C we are proving lower bounds against. This criterion 
is necessary: / ^ C if and only if there is a trivial property, true of only /, distinguishing / from all 
functions computable in C.) In this paper, we study alternative ways to characterize Natural Proofs and 
their relatives as certain circuit lower bound problems, and give several applications. There are multiple 
competing intuitions about the meaning of Natural Proofs. We wish to rigorously understand the extent to 
which the Razborov-Rudich framework relates to our ability to prove lower bounds in general. 

NEXP lower bounds are constructive and useful Some relationships can be easily seen. Recall EXP 
and NEXP are the exponential-time versions of P and NP. If EXP <f_ C, one can obtain a polynomial-time 
(non-large) property useful against C? So, strong enough lower bounds entail constructive useful properties. 
However, a separation like EXP <£_ C is stronger than currently known, for all classes C containing ACC. 
Could lower bounds be proved for larger classes like NEXP, without entering constructive/useful territory? 
In the other direction, could one exhibit a constructive (non-large) property against a small circuit class like 
TC°, without proving a new lower bound against that class? 

The answer to both questions is no. Call a (non-uniform) circuit class C typical if C 6 {AC , ACC, TC°, 
NC 1 , NC, P/poly}. 3 For any typical C, a property V of Boolean functions is said to be useful against C if, 
for all k, there are infinitely many n such that 

• V{f) is true for at least one / : {0, l} n -> {0, 1}, and 

• V(g) is false for all g : {0, l} n — > {0, 1} having n k size C-circuits. 

In other words, V distinguishes some function from all easy functions. We prove: 

Theorem 1 For all typical C, NEXP (£ C if and only if there is a polynomial-time property of Boolean 
functions that is usefid against C. 

Theorem 1 helps explain why it is difficult to prove even NEXP circuit lower bounds: any NEXP lower 
bound must meet precisely two of the three conditions of Natural Proofs (constructivity and usefulness). 4 

One can make a heuristic argument that the recent proof of NEXP <f_ ACC ([29]) evades Natural Proofs 
by being non-constructive. Intuitively, the proof uses an ACC Circuit SAT algorithm that only mildly im- 
proves over brute force, so it runs too slowly to obtain a polytime property useful against ACC. Theorem 1 

'See [1] for a discussion with many views on these questions. 

2 Define A(T) to accept its 2™ -bit input T if and only if T is the truth table of a function that is complete for E = TIME[2° (n) ]. 
A runs in poly(2 n ) time and rejects all T with C circuits, assuming EXP <(_ C. 

3 For simplicity, in this paper we mostly restrict ourselves to these "typical" classes; however it will be clear from the proofs that 
we only rely on a few properties of these classes, and more general statements can be made. 

4 One may also wonder if non-constructive large properties imply any new circuit lower bounds. This question does not seem 
to be nearly as interesting. For one, there are already coNP-natural properties useful against P/poly (simply try all possible small 
circuits in parallel), and they don't imply anything we don't already know. So anything coNP-constructive or worse is basically 
uninformative (without further information about the property). On the other hand, slightly more constructive properties, such as 
NP-natural ones, seem unlikely [25]. 
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shows that, in fact, constructivity is necessary. Moreover, the proof of Theorem 1 yields an explicit property 
useful against ACC. 

The techniques used in Theorem 1 can be applied, along with several other ideas, to prove new super- 
polynomial lower bounds against ACC. Recall that NE = NTIME[2°( n )] and coNE = coNTIME[2°( n )]. 

Theorem 2 NE n coNE does not have ACC circuits ofn logn size. 5 

This lower bound is intriguing not just because NE n coNE C NEXP, but also because it necessarily 
must be proved differently. The known proof of NEXP (f. ACC works for the class NEXP because there is 
a tight time hierarchy for nondeterminism [30]. However, the NTIME n coNTIME classes are not known 
to have any such hierarchy! (They are among the "semantic" classes, which are generally not known to 
have complete languages or nice time hierarchies.) Interestingly, our proof of Theorem 2 crucially uses the 
previous lower bound framework against NEXP, and bootstraps off of it, via Theorem 1 and a modification 
of the NEXP <f_ ACC lower bound. Indeed, it follows from our arguments (building on [28, 29]) that the 
lower bound consequences of non-trivial circuit SAT algorithms can be strengthened, in the following sense: 

Theorem 3 Let C be typical. Suppose the satisfiability problem for 

n O(\og c n)_ size 

C circuits can be solved 
in 0(2 n /n 10 ) time, for all constants c. Then NE n coNE does not have n sn -size C circuits. 

Theorem 4 Suppose we can approximate the acceptance probability of any given n°( logC n >-size circuit with 
n inputs to within 1/6, for all c, in 0(2 n /n 10 ) time (even nondeterministically). Then NE fl coNE does not 
have n} ogn -size circuits. 

Natural Proofs vs derandomization Given Theorem 1, it is natural to wonder if full-strength natural 
properties are equivalent to some circuit lower bound problems. If so, such lower bounds should be consid- 
ered unlikely \ To set up the discussion, let RE = RTIME[2 M] and ZPE = ZPTIME[2°( n )]; that is, RE is 
the class of languages solvable in 2°( n ) randomized time with one-sided error, and ZPE is the corresponding 
class with zero error (i.e., expected 2°^ running time). 

For a typical circuit class C, we informally say that RE (respectively, ZPE) has C seeds if, for every 
predicate defining a language in the respective complexity class, there are C circuit families succinctly 
encoding exponential-length "seeds" that correctly decide the predicate. (Formal definitions are given in 
Section 4.) Having C seeds means that the randomized class can be derandomized very strongly: by trying 
all poly-size C circuits as random seeds, one can decide any predicate from the class in EXP. 

We give a tight correspondence between the existence of such seeds, and the nonexistence of natural 
properties: 

Theorem 5 The following are equivalent: 

1. There are no P '-natural properties useful (respectively, ae-useful 6 ) against C 

2. ZPE has C seeds for almost all (resp., infinitely many) input lengths 

5 We remark that this is not the strongest size lower bound that can be proved, but it is among the cleanest. 
6 Here, ae-useful is just the "almost-everywhere useful" version, where the property is required to distinguish random functions 
from easy ones on almost every input length. 
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Informally, the theorem says that ruling out P-natural properties is equivalent to a strong derandom- 
ization of randomized exponential time, using small circuits to encode exponentially-long random seeds. 
Similarly, we prove that a variant of natural properties is related to succinct "hitting sets" for RE (Theo- 
rem 14). 

It is worth discussing the meaning of these results in a little more detail. Let C, V be appropriate circuit 
classes. Roughly speaking, the key lesson of Natural Proofs [24, 22, 19] is that, if there are P-natural 
properties useful against C, then there are no pseudorandom functions (PRFs) computable in C that fool V 
circuits; namely, there is a statistical test T computable in T> such that, for every function / € C (armed with 
an n-bit initial random seed), the test T with query access to / can distinguish / from a uniform random 
function. Now, if we have a PRF computable in C that can fool V circuits, this PRF can be used to obtain C 
seeds for randomized V circuits with one-sided error. 7 That is, the existence of PRFs implies the existence of 
C seeds, so our consequence in Theorem 5 (of the existence of natural properties) that "no ZPE predicate has 
C seeds" appears stronger than "there are no PRFs." Moreover, this stronger consequence in Theorem 5 (and 
Theorem 14, proved later) yields an implication in the reverse direction: the lack of P-natural properties 
implies strong derandomizations of randomized exponential-size V. 

Theorem 5 also shows that some plausible derandomization problems are as hard as resolving P ^ NP. 
Suppose assuming P = NP, we can give a "canonical" derandomization of ZPE in EXP, along the lines 
of item (2) in Theorem 5. (Clearly if P = NP, we have ZPE C NEXP = EXP, but we ask for a special 
derandomization here.) By Theorem 5, there are no P-natural properties useful against P/poly. However, 
there are coN P-natural properties of this kind, so P ^ NP would follow(!). 

Unconditional mild derandomizations Understanding the relationships between the randomized com- 
plexity classes ZPP, RP, and BPP is a central problem in modern complexity theory. It is well-known 
that 

P C ZPP = RP n coRP C RP C BPP 

but it is not known if any inclusion is an equality. The ideas behind Theorem 5 can also be applied to prove 
new relations between these classes. 

Theorem 6 Either RTIME[2°( n )] C SIZE[n c ] for some c, or BPP C io-ZPT\ME[2 n£ ]/n £ for all e > 0. 

We have a win-win: either randomized exptime is very easy with non-uniform circuits, or random- 
ized computation with two-sided error has a zero error simulation that dramatically avoids brute-force. 
To appreciate the theorem statement, suppose the first case could be modified to conclude that RP C 
/o-ZPTIME[2 n "]/n e for all e > 0. Then the famous (coRP) problem of Polynomial Identity Testing would 
have a new subexponential-time algorithm, good enough to prove strong NEXP circuit lower bounds. 8 A 
quick corollary of Theorem 6 comes close to achieving this. To simplify notation, we use the SUBEXP 
modifier in a complexity class to abbreviate "2™ £ time, for every e > 0." 

Corollary 1 For some c, RP C /o-ZPSUBEXP/n c . 

'Consider any ©-circuit D that tries to use / as a source of randomness. A C-circuit seed for D can be obtained from a circuit 
computing /: since / fools D, at least one n-bit seed to / will make print 1. 

8 More precisely, the main result of Kabanets and Impagliazzo [17] concerning the derandomization of Polynomial Identity Test- 
ing (PIT) can be extended as follows: if PIT for arithmetic circuits can be solved for infinitely-many circuit sizes in nondeterministic 
subexponential time, then either NEXP <f_ P/poly or the Permanent does not have polynomial-size arithmetic circuits. 
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That is, the error in an RP computation can be removed in subexponential time with fixed-polynomial 
advice, infinitely often. We emphasize that the advice needed is independent of the running times of the RP 

and ZPSUBEXP computations: the RP computation could run in n c time and still need only n c advice to 

1/C CCC 

be simulated in 2 n time. Corollary 1 extends a theorem of Kabanets [15], who gave a simulation of RP 
in pseMfifo-subexponential time with zero error. That is, his simulation is only guaranteed to succeed against 
efficient adversaries which try to generate bad inputs — it does not succeed on all inputs. 

Other applications include a new simulation of Arthur-Merlin games and a new equivalence between 
NEXP = BPP and nontrivial simulations of BPP. 

2 Preliminaries 

For simplicity, all languages are over {0, 1}. We assume knowledge of the basics of complexity the- 
ory [4] such as advice-taking machines, and complexity classes like EXP = TIME[2 n ° (1) ], NEXP = 
NTIME[2 n ° (1) ], AC°[m], ACC, and so on. We use SIZE[s(n)] to denote the class of languages recognized 
by a (non-uniform) s(n)-size circuit family. We also use the (standard) "subexponential-time" notation 
SUBEXP = n e>0 TIME [ 2 ° {n£) ]- ( So for example, NSUBEXP refers to the class of languages accepted 
in nondeterministic 2™ £ time, for all e > 0.) When we refer to a "typical" circuit class (AC , ACC, TC°, 
NC 1 , NC, or P/poly}), we will always assume the class is non-uniform, unless otherwise specified. Some 
familiarity with prior work connecting SAT algorithms and circuit lower bounds [28, 29] would be helpful, 
but this paper is mostly self-contained. 

We will use infinitely-often classes: for a complexity class C, io-C is the class of languages L such that 
there is an V G C where, for infinitely many n, L n {0, l} n = L' D {0, l} n . 

We also use advice classes: for a class C and a function a{n), C/a(n) is the class of languages L such 
that there is an V G C and an arbitrary function / : N —¥ {0, 1}* with |/(n)| < a(n) for all x, such that 
L = {x | (x, f(\x\)) G L'}. That is, the arbitrary advice string f(n) can be used to solve all n-bit instances 
within class C. 

Some particular notation and conventions will be useful for this paper. For any circuit C{x\,X2, ■ ■ ■ , x n ), 
i < j, and a\, . . . , a n G {0, 1}, the notation C(a\, . . . , cjj, •, aj, . . . , a n ) represents the circuit with j — i — 1 
inputs obtained by assigning the input x q to a q , for all q G [1, i] U [j, n]. In general, • is used to denote free 
unassigned inputs to the circuit. 

For a Boolean function / : {0, 1}™ — s> {0, 1}, the truth table of / is defined to be 

«(/) :=/(yi)/(j/2)---/(ite»), 

and the truth table of a circuit is simply the truth table of the function defining it. For binary strings with 
lengths that are not powers of two, we use the following encoding convention. Let T be a binary string, let 
k = [log 2 \T\~\, and let yi, . . . , y 2 k G {0, l} fc be the list of fc-bit strings in lex order. The function encoded 
by T, denoted as fr, is the function satisfying M(/t) = TO 2 ~\ T \. 

The size of a circuit is its number of gates. The circuit complexity of an arbitrary string (and hence, a 
function) takes some care to properly define, based on the circuit model. For the unrestricted model, the 
circuit complexity ofT, denoted as CC{T), is simply the minimum size of any circuit computing fx- For 
a depth-bounded circuit model, where a depth function must be specified prior to giving the circuit family, 
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the appropriate measure is the depth-d circuit complexity ofT, denoted as CC d {T), which is the minimum 
size of any depth-d circuit computing /y. (Note that, even for circuit classes like NC 1 , we have to specify 
a depth upper bound clog n for some constant c.) For the class ACC, we must specify a modulus m for the 
MOD gates, as well as a depth bound, so when considering ACC circuit complexity, we look at the depth-d 
mod-m circuit complexity ofT, CC d;m (T), for fixed d and m. 

A simple fact about the circuit complexities of truth tables and their substrings will be very useful: 

Proposition 1 Suppose T = T\ ■ ■ ■ T 2 k is a string of length 2 k+e , where T\ , . . . , T 2 k each have length 2 . 
Then CC(Ti) < CC{T), CC d (Ti) < CC d {T), and CC d , m (Ti) < CC d , m (T). 



Proof. Given a circuit C of size s for fr, a circuit for /y is obtained by substituting values for the first k 
inputs of C. This yields a circuit of size at most s. □ 

We will sometimes need a more general claim: for any string T, the circuit complexity of an arbitrary 
substring of T can be bounded via the circuit complexity of T. 



Lemma 1 There is a universal c > 1 such that the following holds. Let T be a binary string, and let S be 
any substring ofT. Then for all d and m, CC(f s ) < CC{f T ) + (clog \T\), CC d {f s ) < CC d+c (f T ) + 
(clog ITI)^ 1 ), and CC d , m (fs) < CC d+c , m {fr) + (clog \T\f+°V. 



Proof. Let d be sufficiently large in the following. Let k be the minimum integer satisfying 2 k > \T\, so the 
Boolean function /y representing T has truth table TO 2 Suppose C is a size-s depth-d circuit for /y. 
Let S be a substring of T = t x ■ ■ ■ t 2 h € {0, l} 2k , and let A, B £ {1, . . . , 2 k } be such that S = t A - ■ -t B . 
Let I < k be a minimum integer which satisfies 2 e > B — A. Our goal is to construct a small circuit D with 
£ inputs and truth table S0 2e ~( B ~ A \ 

Let xi, . . . , x 2 i be the ^-bit strings in lex order. The desired circuit D on input xi can be implemented 
as follows: Compute i + A. If i + A < B — A then output C{xi + a), otherwise output 0. To bound 
the size of D, first note there are depth-c' circuits of at most c' ■ n log* n size for addition of two n-bit 
numbers [9]. Therefore in depth-c' and size at most d ■ A; log* k we can, given input Xi of length £, output 
i + A. Determining ifi + A<B — A can be done with (c' • ^)-size depth-c' circuits. Therefore D can either 
be implemented as a circuit of size at most s + c'{{k log* k) +£ + 1) and depth 2c' + d, or as an (unrestricted 
depth) circuit of size at most s + c'(k + t + 1). To complete the proof, let c > 3c'. □ 

We will use the following strong construction of pseudorandom generators from hard functions: 



Theorem 7 (Umans [27]) There is a universal constant g and a function G : {0, 1}* x {0, 1}* — > {0, 1}* 
such that, for all s and Y satisfying CCiY) > s 9 , and for all circuits C of size s, 



Pr [C(G(Y,x)) = l]- Pr [C(x) = 1] 

zG{0,l}9 lo sl y l xG{0,l} s 



< 1/s. 



Furthermore, G is computable in poly(|y|) time. 
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Natural Proofs A property of Boolean functions V is a subset of the set of all Boolean functions. Let T 
be a complexity class and let C be a circuit class (typically, T = P and C = P/poly). A T -natural property 
useful against C is a property of Boolean functions V that satisfies the axioms: 

(Constructivity) V is decidable in T, 

(Largeness) for all n, V contains a l/2°( n ) fraction of all ra-bit inputs, 

(Usefulness) Let / = {f n } be a sequence of functions {f n } such that f n £ V for all n. Then for all 
k and infinitely many n, f n does not have re fc -size C-circuits. 9 

Let / = {/„ : {0, 1}™ — > {0, 1}} be a sequence of Boolean functions. A T-natural proof that / ^ 
C establishes the existence of a T-natural property V useful against C such that V(f n ) = 1 for all n. 
Razborov and Rudich proved that any P/poly -natural property useful against P/poly could break all strong 
pseudorandom generator candidates in P/poly. More generally, P /poly-natural properties useful against 
typical C C P/poly imply there are no strong pseudorandom functions in C (but such functions are believed 
to exist, even when C = TC° [22]). 

2.1 Related Work 

Equivalences between algorithms & lower bounds Some of our results are equivalences between algo- 
rithm design problems and circuit lower bounds. Equivalences between derandomization hypotheses and 
circuit lower bounds have been known for some time, and recently there has been an increase in results 
of this form. Nisan and Wigderson [23] famously proved an equivalence between "approximate" circuit 
lower bounds and the existence of pseudorandom generators. Impagliazzo and Wigderson [13] prove that 
BPP ^ EXP implies deterministic subexponential-time heuristic algorithms for BPP (the simulation suc- 
ceeds on most inputs drawn from an efficiently samplable distribution, for infinitely many input lengths). 
As the opposite direction can be shown to hold, this is actually an equivalence. (Impagliazzo, Kabanets, and 
Wigderson [12] proved another such equivalence, which we discuss below.) Two more recent examples are 
Jansen and Santhanam [14], who give an equivalence between nontrivial algorithms for polynomial identity 
testing and lower bounds for the algebraic version of NEXP, and Aydinlioglu and Van Melkebeek [5], who 
give an equivalence between X^-simulations of Arthur-Merlin games and circuit lower bounds for X2EXP. 

The work of IKW Impagliazzo-Kabanets-Wigderson [12] proved a theorem similar to one direction of 
Theorem 1, showing that an NP-natural property (without largeness) useful against P/poly implies NEXP <f_ 
P/poly. Allender [2] proved that there is a (non-large) property computable in NP useful against P/poly if 
and only if there is such a property in uniform AC . Hence his equivalence implies, at least for C = P/poly, 
that the "polynomial-time" guarantee of Theorem 1 can be relaxed to "AC ." 

IKW [12] also give an equivalence between NEXP lower bounds and an algorithmic problem: NEXP <f_ 
P/poly if and only if the acceptance probability of any circuit can be approximated, for infinitely many 
circuit sizes, in nondeterministic subexponential time with subpolynomial advice. The major difference 
between their equivalence and Theorem 1 is the underlying computational problems, along with the algo- 
rithmic guarantees: they study subexponential-time algorithms for approximating acceptance probabilities, 

'Note that some papers replace 'infinitely many' with 'almost every'; in this paper, we call that version ae-usefulness. 



1 



while we study P-useful properties. Moreover, their equivalence is less general with respect to circuit 
classes; for example, it is not known how to prove an analogue of their equivalence for ACC. 

IKW posed the interesting open problem: does the existence of a P-natural property useful against 
P/poly imply EXP (jL P/poly? Our work shows that the absence of a P-natural property implies new lower 
bounds: 

Claim 1 If there is no V -natural property useful against P/poly, then NP 7^ ZPP. 

In brief, NP = ZPP yields a ZP P-natural property, and the ideas behind Theorem 5 can be used to "deran- 
domize" natural properties generically, yielding a P-natural one (see Appendix A for details). Therefore, an 
affirmative solution to the IKW problem would yield a proof that EXP 7^ ZPP. 

Almost-Natural Proofs Chow [10] has shown that if strong pseudorandom generators do exist, then there 
is a proof of NP <f_ P/poly that is almost-natural in that the fraction of inputs in largeness is relaxed from 
1 /2°( n ) to 1 /2 riP ° ly(log n) . Hence the Natural Proofs barrier was known to be somewhat sensitive to relaxations 
of largeness. Chow also proved relevant unconditional results: for example, there exists a SIZEfO(n)]- 

natural property that is i/2 n(logn) -large and useful against P/poly. Theorem 1 shows that if SIZE[0(ra)] 
could be replaced with P, then NEXP (£_ P/poly follows. 

3 NEXP Lower Bounds and Useful Properties 

In this section, we prove: 

Reminder of Theorem 1 For all typical C, NEXP <f_ C if and only if there is a polynomial-time property of 
Boolean functions that is useful against C. 

One piece of the proof is a connection between small circuits for NEXP and small circuits encoding 
witnesses to NEXP languages. This connection was first explored by Impagliazzo, Kabanets, and Wigder- 
son [12], and extended slightly in the author's prior work [28, 29]. We first need a definition of what it 
means for a language (and a complexity class) to have small circuits encoding witnesses. 

Definition 1 Let L € NTIME[i(n)] where t(n) > n is constructible, and let C be a circuit class. An 
algorithm V(x, y) is a predicate for L ifV runs in time 0(\y\) + t(\x\) and for all strings x, 

x € L <4=>- there is a y of length 0(t(n)) (a witness for x) such that V(x, y) accepts. 

We denote L{V) to be the language accepted by V. V has C witnesses of size s(n) if for all strings x, if 
x 6 L then there is a C-circuit C x of size at most s(n) such that V(x, C x (-)) accepts. L has C witnesses of 
polynomial size if for all predicates V for L, there is a polynomial p(n) such that V has C witnesses of size 
0(p(n)). l ° NTIME[t(n)] has C witnesses if for every infinite language L, L has C witnesses of polynomial 
size. 

10 For circuit classes C where the depth d and/or modulus m may be bounded, we also quantify this d and m simultaneously with 
the size parameter p(n). That is, the depth, size, and modulus parameters are chosen prior to choosing an input, as usual. 
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The above definition allows, for every x, a different circuit C x encoding a witness for x. We will also 
consider a stronger notion of oblivious witnesses, where a single circuit C n encodes witnesses for all x G L 
of length n. 

Definition 2 Let L G NTIME[i(n)], let C be a circuit class, and let V be a predicate for L. L has oblivious 
C witnesses of size s(n) if for every predicate V for L, there is a C circuit family {C n } of size s(n) such 
that for all x G {0,1}*, if x G L then V(x, tt(C\ x \(x, •)) accepts. 11 Finally, NTIME[t(n)] has oblivious C 
witnesses if every infinite L G NTIME[i(n)] has oblivious C witnesses. 

We first prove an equivalence between the existence of small circuits for NEXP and small circuits for 
NEXP witnesses, in both the oblivious and normal senses. Let C be a typical circuit class. 

Theorem 8 The following are equivalent: 

(1) NEXP C C 

(2) NEXP has C witnesses of polynomial size 

(3) NEXP has oblivious C witnesses of polynomial size 

Proof. (1) => (2) Impagliazzo, Kabanets, and Wigderson [12] proved this direction for C = P/poly. The 
other cases of C were proved in prior work [28, 29]. 

(2) (3) Assume NEXP has C witnesses of polynomial size. Let V(x, y) be an NEXP predicate that 
(without loss of generality) accepts witnesses y of length exactly 2^ . We will construct a C-circuit family 
{Cn} such that x G L if and only if V{x,tt(C\ x \{x, •))) accepts (recall tt(C\ x \{x, ■)) is the truth table of 
the circuit C\ x \ with x hard-coded and the remaining inputs are free). The idea is to construct a new verifier 
that "merges" witnesses for all inputs of a given length into a single witness. (This theme will reappear 
throughout the paper.) 

Let si,..., x 2 n be the list of strings of length n in lexicographical order. Define a new predicate V 
which takes a pair (x, q) where x G {0, l} n and q = 0, . . . , 2 n , along with y of length 2 n+n : 

V'((x,q),y): Accept if and only if y = z\ ■ ■ ■ z 2 \x\, where for all i, Zi G {0, l} 2 ' 1 ' , exactly 
2 n — q of the Zi strings equals the all-zeroes string, and for all other q strings Zj, V(xj, zj) accepts. 

V runs in time exponential in \x\; by assumption, V' has C witnesses of polynomial size. Observe that 
the computation of V' does not depend on the input x. 

To obtain oblivious C witnesses for V, let q n be the number of x of length n such that x G L(V). Then 
for every y" such that V'((x',q n ),y") accepts, the string y" must encode a valid witnesses Z{ for every 
Xi G L(V), and all-zero strings for every Xj L(V). By assumption, there is a circuit Cf x i^ such that 
C(z',q n ) (i) outputs the ith bit of y" . This circuit C( x ' i9n ) yields the desired witness circuit: indeed, the circuit 
D n (x,j) := Cr x > ,q n ){x o j) (where x o j denotes the concatenation of x and j as binary strings) prints the 
jth bit of a valid witness for x (or 0, if x ^ L(V)). 

(3) (1) Assume NEXP has oblivious C witnesses. Let M be a nondeterministic exponential-time 
machine. We want to give a C-circuit family recognizing L(M). First, define the NEXP predicate 

"That is, the truth table of C\ x \ with x hard-coded is a valid witness for x. 
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Vk(x, y): For all circuits C of size \x\ k + k, 

If tt(C) encodes an accepting computation history of M(x), then 
accept if and only if the first bit of y is 1. 

End for 

Accept if and only if the first bit of y is 0. 

By assumption, there is a A: such that accepting computation histories of M on all length n inputs can 
be encoded with a single (n k + /c)-size C circuit family. For such a k, Vk will run in 2°( n ) time and will 
always find a circuit C encoding an accepting computation history of M(x), when x 6 L{M). Therefore, 
Vk(x, y) accepts if and only if 

[(first bit of y = 1) A (x G L(M))} V [(first bit of y = 0) A (x <£ L(M))\. 

Because Vk is an NEXP predicate, we can apply the assumption again to Vk itself, meaning there is a 
C-circuit family {C n } encoding witnesses for Vk obliviously. This family can be easily used to compute 
L(M): define the circuit D n for n-bit instances of L(M) to output the first bit of the witness encoded by 

C n (x, •)■ □ 

Next, we prove a tight relation between witnesses for N E computations and constructive useful proper- 
ties. (This equivalence will be useful for proving new consequences later.) Here, the typical circuit class C 
does not have to be polynomial-size bounded; the size function s(n) can be any reasonable function in the 
range [n 2 , 2 n / (2n)] (for example). 



Theorem 9 For all s(n), the following are equivalent: 

1. There is a c € (0, 1] such that NTIME[2°( n )] does not have s(cn) size witness circuits from C. 

2. There is a c S (0, 1] and a F '-computable property that is useful against C-circuits of size at most 
s(cn). 12 



Proof. (2) => (1) Let A be a poly(?7,)-time algorithm which takes n bits of input T and is useful against 
size-s(cn) C. Define a machine M(x, T) which rejects if |T| ^ 2 2 l x l, otherwise, it partitions T into strings 
Ti, . . . , T 2 \ x \ of 2^1 bits each, and accepts iff A(T X ) accepts, where T x is the xth block of T (treating x as 
an integer from 1 to 2^). 

Define L = {x | (3 T : |T| = 2^)[M(x,T) accepts]}. Clearly L G NTIME[2°( n )]. Suppose for 
contradiction that l\ITIME[2°( n )] has s(dn) size witnesses, for all d E (0, 1]. Then for almost every £, if 
x E L and |x| = I then there is a circuit C with 21 inputs and size at most s{d£), such that M(x, tt(C)) 
accepts. By definition of M, this means the xth block of tt(C) (call it T x ) is accepted by A. By Proposition 1, 
the C-circuit complexity of T x is at most s(d£) (with the same depth/modulus), as it is a substring of tt(C). 
However, by assumption on A, there are infinitely many i such that T x must have circuit complexity greater 
than s(c£), a contradiction when d < c. 

(1) => (2) Suppose NTIME[2°( n ^] does not have s(n)-size witness circuits. Let V be such a predicate 
(running in NTIME[2 cn ] for some c > 1) that does not have s(n)-size witnesses, but L(V) is infinite. Then 

l2 For circuit classes C with depth bound d, this d will be universally quantified after c. So for example, there is a c such that for 
all constant d, NTIME[2°' n '] does not have s(cn) size depth-d AC°[6] witnesses, if and only if there is a c such that for all d, there 
is a P-computable property useful against depth-rf AC [6] circuits of size s(cn). 
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there is an infinite subsequence of inputs {x-} such that x\ € L{V), but for every x\ and every y such that 
M(x' i ,y) accepts, y requires s(|a^|) size circuits to encode. 

We give a polynomial-time algorithm A that is useful. The trick (similar to the (2) to (3) direction of 
Theorem 8) is to "merge" all the witnesses for a given length into one string. When A is given a string y, if 
y does not have the form y\ ■ ■ ■ y 2 «01 for some k = 0, . . . , 2 e and £ where every \yi\ = 2 c£ , then A rejects. 
Otherwise, A extracts k from z by counting the trailing l's on the end of the string, and accepts if and only 
if k equals the number of i = 1, . . . , 2 l such that V{xi,yi) accepts (here, xi is the ith £-hit string in lex 
order). 

Suppose the k extracted from A is exactly the number of inputs of length £ accepted by V. Then, for 
every x of length £ accepted by V, the corresponding substring yi in y is a witness for x. If there is an i, j 
such that Xj = x\ (the kth ^-bit string equals the ith string in the infinite sequence), then every y' that is a 
witness for Xj requires at least s(£)-size circuits. Hence the substring yj requires s(£)-size circuits, so by 
Lemma 1, y requires circuits of size s(£) — > s(£)/2 (with depth possibly smaller, by a universal 

additive constant). 

Notice that, for each £, and every possible k = 0, . . . , 2r, there is precisely one input length, namely 
n = 2( c+1 )^ + k + 1, for which this value of k will be considered on ^-bit strings. Therefore, on those input 
lengths n for which k equals the number of inputs of length £ accepted by V, A is useful against size-s(^) /2; 
in terms of the input length n, this quantity is at least s(d log n) for some constant d > 0. There are infinitely 
many such n, so this completes the proof. □ 

Using complete languages for NEXP, one can obtain an explicit property in P that is useful against 
C circuits, if there is any constructive useful property. This universality means that, if there are multiple 
constructive properties that are useful against various circuit size functions, then there is one constructive 
property useful against all these size functions. 

Theorem 10 Let {sfc(n)} be an infinite family of functions such that for all k, there is a P '-computable 
property that is useful against all C-circuits of Sk (n) size. Then there is a single property P* such that, 
for all k, there is a c > such that P* is useful against all C-circuits of Sfc(cn) size. 13 

Proof. Let b(n) denote the nth string of {0, 1}* in lexicographical order; note that \b(n)\ < log 2 (n + 1). 
The Succinct Halting problem consists of all triples (M, x, b(n)) such that the nondeterministic TM M 
accepts x within at most n steps. Define the algorithm 

HlSTORY(y): Compute z = b(\y\). If z does not have the form (M,x,b(n)}, reject. Accept 
if and only if there is a prefix y' of y with length equal to a power of two such that y' encodes an 
accepting computation history to z G SUCCINCTHALTING. 

Observe that HISTORY is implementable in polynomial time. The theorem follows from the claim: 

Claim 2 HISTORY is useful against C circuits of size s{cn) for some c > if and only if there is some 
P-time property that is useful against C circuits of size s(n). 

To see why the theorem follows, observe that if we have infinitely many properties P^, each of which 
are useful against C circuits of Sfc(n) size, then for every k, HISTORY will be useful against Sfc(n) size C 
circuits. 

13 For depth-bounded/modulus-bounded circuit classes C, an analogous statement holds where we quantify not only over k but 
also the depth d and modulus m. 
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One direction is obvious. For the other, suppose there is a polynomial-time property useful against C- 
circuits of size s(n). By Theorem 9, l\ITIME[2°( n )] does not have s(dn) size witnesses from C for some 
constant d. Let V be a predicate running in time 2 kn that does not have s(e£n)-size C witnesses, and let M 
be the corresponding nondeterministic machine which, on x, guesses a y and accepts iff V(x, y) accepts. It 
follows that there are infinitely many instances of SUCCINCTHALTING of the form (M, x, b(2 k ^)) that do 
not have C witnesses of size s(cn) for some constant c. Therefore, there are infinitely many Z{ = (Mj, ajj, n,) 
in SUCCINCTHALTING, where every accepting computation history y' of Mj(xj) has greater than s(cn)- 
size C-circuit complexity. Then for all n such that Z{ = b{n) for some i, there is a y of length n such 
that HlSTORY(y) accepts but for all y" which encode functions with C-circuits of s(cn)-size, HlSTORY(y") 
rejects (by Proposition 1; note y" has length equal to a power of two). Hence HISTORY is useful against C 
circuits of size s(cn). This concludes the proof of the theorem. □ 

Putting it all together, we obtain Theorem 1 : 

Proof of Theorem 1. Let C be a typical class (of polynomial-size circuits). By Theorem 8, we have 
NEXP (£_ C if and only if for every k, NEXP does not have C witnesses of n k size. 

Setting s(n) = n k for arbitrary k in Theorem 9, we infer that for every k, we have the equivalence: 
NEXP does not have C witnesses of n k size if and only if there is c > and a P-computable property that is 
useful against all C-circuits of size at most (cn) k . 

Applying Theorem 10, we conclude that NEXP <f_ C if and only if there is a P-computable property such 
that, for all k, it is useful against all C-circuits of size at most n k . □ 



3.1 New ACC Lower Bounds 

In this section, we prove new lower bounds against ACC. Our approach uses a new nondeterministic simula- 
tion of randomized computation (assuming small circuits for ACC). The simulation itself uses several ingre- 
dients. First, we prove an exponential-size lower bound on the sizes of ACC circuits encoding witnesses for 
NTIME[2°( n )]. (Recall that, for NEXP, the best known ACC size lower bounds are only "third-exponential", 
e.g., quasi-polynomial [29].) Second, we use the connection between witness size lower bounds and con- 
structive useful properties of Theorem 9. The third ingredient is a well-known hardness-randomness con- 
nection: from a constructive useful property, we can nondeterministically guess a hard function, verify its 
hardness using the property, then use the hard function to construct a pseudorandom generator. (Here, we 
will need to make an assumption like P C ACC, as it is not known how to convert hardness into pseudoran- 
domness in the ACC setting [26].) 

Theorem 11 For all d, m there is an e > such that NTIME[2°( n )] does not have 2 n " -size d-depth 

AC [m] witnesses. 14 

The proof relies heavily on the NEXP <f_ ACC proof, so we will merely sketch how it is different. 

Proof. (Sketch) Assume NTIME[2°( n )] has 2 n£ -size ACC witnesses, for all e > 0. We will show that 
the earlier framework [29] can be adapted to still establish a contradiction. First, observe the assumption 
implies that TIME[2°( n )] has 2 n " -size ACC circuits. (The proof is similar to the proof of Theorem 1: for 

l4 The m e(d) factor arises from the ACC-SAT algorithm in [29], which in turn comes from Beigel and Tarui's simulation of ACC 
in SYM-AND [7]. 
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any given exponential-time algorithm A, one can set up an NEXP predicate that only accepts its input of 
length n if the witness is a truth table for the 2 n -bit function computed by A on n-bit inputs. Then, a witness 
circuit for this x is a circuit for the entire function on n bits.) Therefore (by Lemma 3.1 in [29]) there is 
a nondeterministic 2 n ~ n time algorithm A (where 5 depends on the depth and modulus of ACC circuits 
for Circuit Evaluation) that, given any circuit C of size and n inputs, A generates an equivalent ACC 
circuit C' of 2" e size, for all e > 0. (More precisely, there is some computation path on which A generates 
such a circuit, and on every path, it either prints such a circuit or outputs fail.) 

We will simulate every L G NTIME[2 n ] in nondeterministic time 2 n ~ n * (this will contradict the nonde- 
terministic time hierarchy of Zak [30]). Given an instance x of L, we first reduce L to the NEXP-complete 
SUCCINCT 3SAT problem using an efficient polynomial-time reduction. This yields an unrestricted circuit 
D of size and n + 0(log n) inputs with truth table equal to a formula F, such that F is satisfiable if 
and only if x G L. We run algorithm A on D to obtain an equivalent 2 n " size ACC circuit D' . Then we 
guess a 2™ e size ACC circuit E with truth table equal to a satisfying assignment for F. (If x G L, then such 
a circuit exists, by assumption.) By combining copies of D' and copies of E, we can obtain a single ACC 
circuit C with n + 0(log n) inputs which is unsatisfiable if and only if E encodes a satisfying assignment for 
F. By calling a nontrivial satisfiability algorithm for ACC, we get a nondeterministic 2 n ~™ time simulation 
for every L, a contradiction. □ 

Applying Theorem 9 and its corollary to the lower bound of Theorem 11, we can conclude: 

Corollary 2 There is a P -computable property that is useful against all depth-d AC°[m] circuits of size at 
most T l ° , for e = l/m ^. 

Hence there is an efficient way of distinguishing some functions from all functions computable with 
subexponential-size ACC circuits. Let CAPPbe the problem: given a circuit C, output p G [0, 1] satisfying 

\Pr x [C(x) = 1] -p\ < 1/6. 

That is, we wish to approximate the acceptance probability of C to within 1/6. We can give a quasi- 
polynomial time nondeterministic algorithm for CAPP, assuming P is in quasi-polynomial size ACC. 

Theorem 12 Suppose P has ACC circuits of size n logn . Then there is a constant c such that for infinitely 
many sizes s, CAPP for size s circuits is computable in nondeterministic 2( logs ) c time. 

Theorem 12 is a surprisingly strong consequence: given that NEXP (f_ ACC, one would expect only a 
2°( nE )-time algorithm for CAPP, with n e bits of advice. (Indeed, from the results of IKW [12] one can derive 
such an algorithm, assuming P C ACC.) Recall a unary language is a subset of {1™ | n G N} C {0, 1}*. 
The proof of Theorem 1 1 also has the following consequence: 

Corollary 3 If P has ACC circuits of n} ogn size, then for all d, m there is an e such that there are unary 
languages in NTIME[2 n ] without 2 n " -size d-depth AC°[m] witnesses. 

Proof. The tight nondeterministic time hierarchy of Zak [30] holds also for unary languages — that is, there 
isaunaryL G NTIME[2 n ] \ NTIME[2 n /n 10 ]. So assume (for a contradiction to this hierarchy) that all unary 
languages in NTIME[2 n ] have 2 n£ size witnesses for every e > 0. This says that, for every predicate V for 
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any unary language L £ NTI M E [2 n ] , every 1" 6 L has a witness y with 2 n -size circuit complexity. Choose 
a predicate V that reduces a given unary L to a SUCCINCT3SAT instance, then checks that its witness is a 
SAT assignment to the instance; by assumption, such SAT assignments must have circuit complexity at most 
2 n \ for almost all n. By guessing such a circuit and assuming P has n logn -size ACC circuits, the remainder 
of the proof of Theorem 11 goes through: the simulation of arbitrary L in NTIME[2 n ~ n ] works and yields 
a contradiction. □ 

Corollary 3 allows us to strengthen Corollary 2, to yield a "nondeterministically constructive" and useful 
property against ACC. 

Proof of Theorem 12. First we claim that, if P has n logn size ACC circuits, then there is a d* and m* such 
that every Boolean function / with unrestricted circuits of size S has depth-d* AC [m*] circuits of size at 
most S logS . To see this, consider the CIRCUIT EVALUATION problem: given a circuit C and an input x, 
does C(x) = 1? Assuming P is in n logn ACC, this problem has a depfh-<i* 74C°[m*] circuit family {D n } 
of n logn size, for some fixed d* and m*. Therefore, by plugging in the description of any circuit C of size S 
into the input of the appropriate ACC circuit -Do(S)> we g et an ACC circuit of fixed modulus and depth that 
is equivalent to C and has size 0(S logS ). 

By Corollary 3, there is an e and a unary L in NTIME[2 n ] that does not have 2™ e size AC°[m*] witnesses 
of depth d*. By the previous paragraph (and assuming P is in n logn -size ACC), it follows that L does not 
have witnesses encoded with 2 n ~ /2 -size unrestricted circuits. (Letting S l ° sS = 2 n " , we find that S = 2 n£/2 .) 
Let V be a predicate for L that lacks such witnesses, and let g be the constant in the pseudorandom generator 
of Theorem 7. Consider the nondeterministic algorithm P which, on input I s , sets n = (g log s) 2 / £ , guesses 
a string Y of 2™ length, and outputs Y if V(l n ,Y) accepts (otherwise, P outputs reject). For infinitely 
many s, P(l s ) nondeterministically generates strings Y of 2( 9logs ) 2/£ length that do not have s 9 = 2 n£/2 
size circuits: as there is an infinite set of {rij} such that all witnesses to \ Ui have circuit complexity at least 
2("») e 2 5 there is an infinite set {si} such that P(l St ) computes nj = (glogSj) 2 / e and generates Y which 
does not have (si) g = 2^ Ul ^ /2 size circuits. 

Given a circuit C of size s, our nondeterministic simulation runs P to generate Y. (If P rejects, the 
simulation rejects.) Applying Theorem 7, Y can be used to construct a poly(|y|)-time PRG G(Y,-) : 
{0, l}9 lo s \ Y \ -> {0, 1} S which fools circuits of size s. By trying all \Y\o < 2°(( logs ) 2/e ) inputs to G Y , we 
can approximate the acceptance probability of a size-s circuit in 2°^ logs ^ 2/ ^ time. As e depended only on 
d* and m*, which are both constants, we can set c = 3/e to complete the proof. □ 

Now we turn to proving lower bounds for the class NE n coNE. We will need the following result of 
Miltersen-Vinodchandran-Watanabe [20] extending Babai-Fortnow-Nisan-Wigderson [6]: 

Theorem 13 (Lemma 8, [20]) Let g(n) > 2 n and s{n) > nbe increasing and time constructible. There is 
a constant c> 1 such that TIME[2°( n )] C SIZE[s(n)] TIME[g(n)] C MATIME[s(3 log g{n)) c ]. 

That is, if we assume exponential time has s(n)-size circuits, we can simulate even larger time bounds 
with Merlin-Arthur games. This follows from the proof of EXP C P/poly => EXP = MA ([6]) combined 
with a padding argument. 

Reminder of Theorem 2 NE n coNE does not have ACC circuits ofn logn size. 

Proof. Suppose NE n coNE has n logn -size ACC circuits. We wish to derive a contradiction. Of course the 
assumption implies that TIME[2°( n )] has ro logn -size circuits as well. Applying Theorem 13 with g(n) = 
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2™ 21 ° g " and s(n) = ra logn , we have 

TIME[2 n21 ° gn ] C MATIME[n°( log3n )]. 

By Theorem 12 and assuming that P has ACC circuits of size n logn , there is a constant c and a pseudorandom 
generator with the following properties: for infinitely many circuit sizes s, the generator nondeterministi- 
cally guesses a string Y of length 2( logs ) c , verifies Y in poly(|Y|) deterministic time with a useful property 
P, then uses Y to construct a PRG that runs in poly(|Y|) time deterministically over poly(|Y|) different 
seeds. The poly(|Y|) outputs of length s can then be used to correctly approximate the acceptance proba- 
bility of any size s circuit. 

We can use this generator to fool Merlin-Arthur games on infinitely many circuit sizes, as well as co- 
Merlin- Arthur games. Take a n°( log n )-size circuit C encoding the predicate in a given Merlin- Arthur 
game of that length (C takes an input x, Merlin's string of length re°' log n \ and Arthur's string of length 
n O(iog n)^ an( j 0U tp U t s a bit). On the circuit sizes for which the generator works, we can guess Merlin's 
string m, then use the PRG on C(x, m, •) to simulate Arthur's string and the final outcome. Hence there is 
a constant d such that 

TIME[2 n21 ° s ™] C MATIME[n°( log3n )] C /o-NTIME[n logdn ]. 

As TIME[2 n21os "] is closed under complement, an analogous argument (applied to any machine accepting 
the complement of a given TIME[2 n ° s "] language) implies 

TIME[2 n2 '° sn ] C coMATIME[n°( log3n )] C zo-coNTIME[n logdn ]. 
At this point, we have 

TIME[2 n21 ° sn ] C /o-(NTIMEncoNTIME)[n logdn ]. 

Assuming every language in NE n coNE has circuits of size n logn , it follows that every language in the class 
/o-(NE fl coNE) has circuits of size n logn for infinitely many input lengths. Therefore 

TIME[2 n21 ° sn ] c io-SIZE[n logn ]. 

But this is a contradiction: for almost every n, by simply enumerating all n logn -size circuits and their 2 n -bit 
truth tables, we can compute the lexicographically first Boolean function on n bits which does not have 
n logn size circuits, in 0(2 n ° s ") time. □ 

We conclude the section by sketching how the above argument can be seen as a more generic result: 

Reminder of Theorem 3 Let C be typical. Suppose the satisfiability problem for C circuits 

can be solved in 0(2 n /n 10 ) time, for all constants c. Then NE fl coNE does not have n} ogn -size C circuits. 

Proof. (Sketch) Suppose satisfiability for C circuits of n°( lo s Cn ) size is in 0(2 n /n 10 ) time (for all c), and 
that NE n coNE has n logn size circuits. By the proof of Theorem 12, assuming P has n logn size C circuits, 
for all e > 0, we obtain a nondeterminstic algorithm N running in 2 2 ° (1 ° s s> time on all circuits of size s 
(for infinitely many s) and outputs a good approximation to the given circuit's acceptance probability. (In 
particular, from the assumptions we can derive a unary language computable in NTIME[2 n ] that does not 
have witness circuits of n logC n size, for every c; this can be used to obtain a nondeterministic algorithm N as 
in Theorem 12, by setting s = n°( logC n ) , solving for n = 2°^ log s) 1/(c+1) ) ^ runnm g the nondeterministic 
algorithm N in 2°W < 2 2 ° (1 ° g£ s) time, where e < l/(c + 1).) 
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By the same argument as in the proof of Theorem 2, we obtain 

TIME[2 n21 ° sn ] C (MATIMEncoMATIME)[n°( log3n )]. 
By applying algorithm N to circuits of size s = n°( logi ™) and setting e <C 1/4, we obtain 
(MATIME n coMATIME)[n° (log3n) ] C /o-(NTIME n coNTIME)[2° (n) ]. 
But the latter class is in /o-SIZE[n logn ] by assumption; we obtain a contradiction as in Theorem 2. □ 

4 Natural Properties and Derandomization 

In this section, we characterize (the nonexistence of) natural properties as a particular sort of derandom- 
ization problem, and exhibit several consequences. Let ZPE = ZPTIME[2°( n )], i.e., the class of lan- 
guages solvable in 2°( n > time with randomness and no error (the machine can output don't know). RE = 
RTIME[2°( n )] is its one-sided -error equivalent. Analogously to Definition 1 , we define a witness notion for 
ZPE as follows: 

Definition 3 Let L G ZPE. A ZPE predicate for L is a procedure M(x, y) that runs in time poly(|y|)-2°(l x D, 
such that on every x 

• The output of M(x, y) is in the set {1, 0, ?}. 

• x G L ==> Pv y [M(x,y) outputs 1] > 2/3, and for all y, M(x,y) € {1,?}. 

• x L ==> Pr y [M(x, y) outputs 0] > 2/3, and for all y, M(x, y) € {0, ?}. 

ZPE has C seeds if for every ZPE predicate M, there is a k such that for all x, there is a C-circuit C x of size 
at most n k + k such that M(x, tt(C x )) / ?. 15 

That is, C seeds for ZPE are succinct encodings of strings that lead to a decision by the algorithm. 
Analogously, we can define RE predicates and the notion of RE having C seeds: RE predicates will accept 
with probability at least 2/3 when x € L, but reject with probability 1 when x £ L. Hence, when RE has C 
seeds, we only require x € L to have small circuits C x encoding witnesses. 

Succinct seeds for zero-error computation are tightly related to uniform natural properties: 
Reminder of Theorem 5 Let C be a polynomial-size typical circuit class. The following are equivalent: 

1. There are no F '-natural properties useful (respectively, ae-useful 16 ) against C 

2. ZPE has C seeds for almost all (resp., infinitely many) input lengths 

I5 For circuit classes where the depth d and/or modulus m may be bounded, we also quantify this d and m simultaneously with 
the size parameter k. That is, the depth, size, and modulus parameters are chosen prior to choosing the circuit family, as usual. 

16 Here, ae-useful is just the "almost-everywhere useful" version, where the property is required to be distinguish random func- 
tions from easy ones on almost every input length. 
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The intuition is that, given a P-natural useful property, its probability of acceptance can be amplified 
(at a mild cost to usefulness), yielding a ZPE predicate accepting random strings with decent probability 
but still lacks small seeds. In the other direction, suppose a ZPE predicate has "bad" inputs that can't be 
decided using small circuits encoding seeds. This implies that a "hitting set" of exponential-length strings, 
sufficient for deciding all inputs of a given length, must have high circuit complexity — otherwise, all strings 
in the set would have low circuit complexity (by Lemma 1), but at least one such string decides even a bad 
input. Checking for a hitting set is then a P-natural, useful property. 

Proof of Theorem 5. (->(1) => _, (2)) Suppose there is a P-natural property ae-useful (resp., useful) against 
C. For some c, d, this is an n c -time algorithm A such that, for almost all n and all k, A accepts at least a 
l/2 dlogn = l/n d fraction of n-bit inputs. Moreover, for almost all n (resp., for infinitely many n) and all 
k, A rejects all n-bit inputs representing truth tables of (logn) fc -size C-circuits. 

Let b(n) denote the nth string in lexicographical order. Let e > be sufficiently small. Define an 
algorithm V: 

V(x, z): If x 7^ b(\z\) then output ?. Partition z into t = \z\ l ~ e / d strings zi, . . . , Zt each of 
length \z\ e / d . If A(zi) accepts for some i, then output 1; else, output ?. 

We claim V is a ZPE predicate for L = {0, 1}*. Consider a z chosen at random. All the Zi are 
independent random variables, and A accepts at least l/\zi\ d strings of that length. So the probability that all 
Zi are among the (1 — l/|z| e ) fraction of strings of length \ z\ £ l d rejected by A, is at most (1 — l/jzl 6 )^ 1 E d < 
e - ' 2 ' 1 £ e/d . For small enough e, this quantity is less than 2/3, so V accepts a random z with at least 2/3 
probability. 

By construction, V accepts (x, z) precisely when x = b(\z\) and some Zi is accepted by A. Hence for 
almost all input lengths \x\ (resp., infinitely many), when V accepts x via witness w, some Zi has C-circuit 
complexity at least (log(\z\ £ ^ d )) k > £l(log k \z\). Therefore by Lemma 1, z itself has C-circuit complexity 
at least SI ((log \z\) k — (log l^l) 1 " 1-0 ^). As this holds for every k, the predicate V does not have C seeds 
infinitely often (respectively, almost everywhere). 

(-■(2) => Suppose there is a ZPE predicate V that does not have C seeds almost everywhere (resp, 

infinitely often). This means that, for all k and for infinitely many (resp., almost all) input lengths m, there 
is an input x of length m such that, for every r satisfying V(x, r) ? (over random r of length 2 cni ), r has 
C-circuit complexity at least (crii) k . (Note c depends on V only.) 

Define a new predicate V'(x, r) that takes random r of length 2 l+cni where i is the smallest integer such 
that 2?ij < 2 , partitions r into 2 e strings {r^} of length 2 cn ' each, and accepts if and only if V(x,ri) ^ ? 
for some i. Any r with this property does not have circuits of size n k , due to Proposition 1 and the fact that 
such an r contains a substring with circuit complexity at least n k . By standard probabilistic arguments, 
it is likely that r encodes a hitting set, i.e., Pr r [(3x € {0, l} ni )(Vi) V(x, rj) = ?] < 1/3. Therefore, a 
randomly chosen r of length 2 ( - c+1 ^ ni is accepted by V, with probability at least 2/3. 

We now define an algorithm A that gives a P-natural property. A takes R of length ./V as input, computes 
the largest rij such that N > 2 i+cn \ sets r to be the first 2 e+cn% bits of R, and partitions r into r/s as above. 
Then A checks for all x of length rij that some V(x, r{) does not output ?. This algorithm A runs in 
poly(A r ) time and accepts at least 1/2 of its inputs. Furthermore, A must reject all strings R with C-circuit 
complexity at most 0(n k ) (i.e., 0((log N) k )), almost everywhere (resp., infinitely often), because if R had 
such circuits, then all n would as well (by Proposition 1). This occurs for every k, so A is a P-natural 
property useful against polynomial-size C circuits. □ 
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To prove a related result for RE predicates, we first need a little more notation. Let V be an RTI M E [2 kn ] 
predicate accepting a language L. For a given input length n, a set S n C {0, l} 2 " is a hitting set for V 
on n if, for all x € L of length n, there is ay G S n such that V(x n ,y) accepts. For a string T of length 
m ■ 2 kn , T encodes a hitting set for V on n if, breaking T into m strings j/i, . . . , y m of equal length, the set 
{yi, . . . , y m } is a hitting set for y on n. 

We also consider another relaxation of naturalness: We say that a property V is io-F '-natural against 
polynomial-size C provided that, for every k and infinitely many n, V accepts at least a l/poly(n) fraction 
of n-bit inputs, and V rejects all n-bit inputs representing functions computable with ((logn) fe + fc)-size 
C-circuits. 17 In the usual notion of natural proofs, largeness holds almost everywhere; here, that is not 
required. We can relate succinctly encoded hitting sets with natural properties as follows: 

Theorem 14 Suppose for all c, RTIME[2°( n )] does not have 0(n 2 )-size hitting sets encoded by n°-size 
circuits. Then for all c, there is an io-P -natural property useful against n c size circuits. 

Proof. The hypothesis says that for every c, there is an RTIME[2°( n )] predicate V c accepting some language 
L with the following property: for every n c -size circuit family {C n }, there are infinitely many n where 
tt(C n ) does not encode a (3(n 2 ) -size hitting set for V c on n. 

We can get an io-natural property computable in P with O(logn) bits of advice, as follows. Given 
an input string Y of length N = 2 kn+2logn , the advice string a encodes the number of inputs of length 
n in L(y c ). Our polynomial-time algorithm partitions Y into y\, . . . ,y 2 2i og n of equal length, and counts 
the number of x of length n < (log N)/k such that V c (x, yi) accepts for some i. If this number equals the 
advice a, then accept else reject. For infinitely many N, this procedure (with the appropriate advice) accepts 
a random string with high probability, and rejects strings encoded by n c -size circuit families, by assumption. 

Now, given an io-P/(log n)-natural A against n c -size circuits, we can convert it into an io-P-natural 
property. For each natural number n we associate the interval I n = [n 2 , (n + l) 2 — 1]; note that the 
collection of I n partitions N. Given input X of length m, our new property A' determines I n such that 
m € I n , and computes a = m — n 2 . Since a £ {0, ... , 2n}, a can be treated as an advice string of length 
(log n) for n-bit inputs; A' takes the first n bits of X, and runs A(x, a). 

For infinitely many input lengths 7ij, A (equipped with the appropriate advice cij) is simultaneously large 
and useful against n c -size circuits. The above shows that each such n» has an associated length mi such that 
the advice a, can be correctly extracted from mi, and A(x, a{) is executed. Hence on these m,i, the property 
A' is both large and useful. Notice that, since the input has increased by a square {mi = G(n 2 )), the strings 
of length m, define functions on only twice as many inputs as rjj. Therefore, when A(x, Oj) accepts (hence 
x has circuit complexity at least (log?ij) c ), by Lemma 1 we can infer that the original input X defines 
a function on at most 21ogmj < 41ogrij bits with circuit complexity at least (log?ij) c — (log ni) 1+ °^ l \ 
Therefore the new property A' is useful against circuits of size (n/4) c . As this condition holds for every c, 
the theorem follows. □ 

The other direction (from io-P-natural to RTIME[2°( n )]) seems difficult to satisfy: it could be that, for 
infinitely many n, the natural property does not obey any nice promise conditions on the number of accepted 
inputs of length n. 

17 As usual, if C is also characterized by a depth d or modulus constraint m, those d and m are quantified alongside k. 
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4.1 Unconditional Mild Derandomizations 



We are now prepared to give some unconditionally-true derandomization results. The first one is: 

Reminder of Theorem 6 Either RTIME[2°( n )] C SIZE[n c ] for some c, or BPP C /o-ZPTIME[2 nE ]/ra £ 
for all e > 0. 

To give intuition for the proof, we compare our strategy with the "easy witness" method of Kabanets [15], 
where he shows that RP can be /wewcio-simulated in /o-ZPTIME[2 n ] (no efficient adversary can generate 
an input on which the simulation fails, almost everywhere). That simulation works as follows: for all e > 0, 
given an RP predicate, try all n e -size circuits and check if any encode a good seed for the predicate. If this 
always happens (against all efficient adversaries), then we can simulate RP in subexponential time. Oth- 
erwise, some efficient algorithm can generate, infinitely often, inputs on which this simulation fails. This 
algorithm generates the truth table of a function that does not have n e -size circuits; this hard function can 
be used to derandomize BPP. 

In order to get a nontrivial simulation that works on all inputs for many lengths, we consider easy hitting 
sets: sets of strings (as in Theorem 14) that contain seeds for all inputs of a given length, encoded by n c -size 
circuits (where c does not have to be tiny, but rather a fixed constant). When such seeds exist for some 
c, we can use 0(n c ) bits of advice to simulate RP deterministically. Otherwise, we apply Theorem 14 to 
obtain an io-P-natural property which can be used (by randomly guessing a hard function) to simulate BPP 
in subexponential time. This allows us to avoid explicit enumeration of all small circuits; instead, we let the 
circuit size exceed the input length, and enumerate over (short) inputs in our natural property. 

Proof of Theorem 6. First, suppose there is a c > 2 so that for every RTIME[2°( n )] predicate V accepting 
a language L, there is an ?i c ~ 1 -size circuit family {C n } such that for almost all n, C n has 0(n) inputs 
and its truth table encodes a hitting set for V on n with 2 21ogn strings. That is, the truth table of C n is a 
string Y of length £ = 2 21ogn • 2 kn for a constant k, with the property that when we break Y into 0(n 2 ) 
equal length strings yi, . . . , y 2 2i og n, the set {t/i} is a hitting set for V on n. Then it follows immediately 
that RTIME[2°( n )] C TIME[2°( n )]/n c , because for almost all lengths n, we can provide the appropriate 
n -size circuit C n as 0(n c ) bits of advice, and recognize L on any n-bit input x by evaluating C on all its 
possible inputs, testing the resulting hitting set of 0(n 2 ) size with x. (We will show later how to strengthen 
this case.) 

If the above supposition is false, that means for every c, there is an RTIME[2°( n )] predicate V c accepting 
some language L with the following property: for every n c -size circuit family {C n }, there are infinitely 
many n such that the truth table of C n does not encode a hitting set for V on n. Theorem 14 says that for all 
c, we can extract an io-P-natural property A c useful against n c size circuits, for all c. In particular, the proof 
of Theorem 14 shows that for all c there are infinitely many n and m 6 [2 n / 3 , 2 3n ] such that A c is useful 
and large on its inputs of length m. So if we want a function / : {0, l}°( n ) — s> {0, 1} that does not have 
n k size circuits, then by setting c = k, providing m as 0{n) bits of advice, and randomly selecting Y of m 
bits, we can generate an / that has guaranteed high circuit complexity, with zero error. 

For every k, we will simulate any language in BPTIME[0(n fe )] (two-sided randomized n k time), as 
follows. Given any k and e > 0, set c = gk/e (where g is the constant in Theorem 7). On input x of 
length n, our ZP simulation will provide advice of length 0{n e ) specifying an input length m = 2 e ( n£ \ It 
then chooses a random string Y of length m, and computes A C (Y). If A C (Y) rejects, then the simulation 
outputs don't know. Otherwise, for infinitely many n, Y is an m = 2 ( nE ) bit string with circuit complexity 
at least (n e ) c > n gk . Applying Theorem 7, Y can be used to construct a PRG Gy : {0, l}3 lo gl y l — >. 
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{0, l} n which fools circuits of size n 3k , where d is a universal constant (independent of e and k). Each 
call to G Y takes poly(|F|) < 2° (n£) time. Trying all \Y\ 9 < 2° (n£) seeds to G Y , we can approximate 
the acceptance probability of a n 3fc -size circuit simulating any BPTIME[0(re fc )] language on n-bit inputs, 
thereby determining acceptance/rejection of any n-bit input. 

Now we have either (1) RTIME[2°( n '] C TIME[2°( n ']/n c for some c, or (2) BPP C /o-ZPTIME[2™ £ ]/n e 
for all e > 0. To complete the proof, we recall that Babai-Fortnow-Nisan-Wigderson [6] proved that if 
BPP <£_ /o-SUBEXP then EXP C P/poly. Therefore, if case (2) does not hold, the first case can be im- 
proved: using a complete language for E, we infer from EXP C P/poly that TIME[2°( n )] C SIZE[n c ] for 
some c, so RTIME[2°M] C SIZE[n c ] for some constant c. □ 

Reminder of Corollary 1 RP C io-ZPSUBEXP /n c for some c. 

Proof. By Theorem 6, there are two cases: (1) RTIME[2°( n )] C SIZE[n c ] for some c, or (2) BPP C 
/o-ZPTIME[2 ne ]/^ efora11 e- In case (1), RP C RTIME[2°( ra )] C TIME[n c ]/n c . In case (2), RP C BPP C 
/o-ZPTIME[2 n£ ]/n £ . □ 

The simulation can be ported over to Arthur-Merlin games: 

Corollary 4 For some c > 1, AM C /o-S 2 SUBEXP/n c . 

Giving a nontrivial relationship between AM and X^P has been open for a long time [11, 5]. The proof 
is very similar: we consider hitting sets for AM computations, succinctly encoded by circuits with oracle 
gates that compute SAT. Either some n c -size SAT-oracle circuits encode hitting sets for AM (in which case, 
AM is in P NP /?i c ), or the ability to test a hitting set for AM becomes a test that a given function does 
not have polynomial-size SAT-oracle circuits (in which case, we can guess a hitting set and verify it, in 
/o-S 2 SUBEXP). 

It is plausible that Corollary 4 could be combined with other results (for example, the work on lower 
bounds against fixed-polynomial advice, of Buhrman-Fortnow-Santhanam [8]) to separate £2 EXP from 
AM. 

Another application of Theorem 6 is an unexpected equivalence between the infamous separation prob- 
lem NEXP ^ BPP and zero-error simulations of BPP. We need one more definition: Heuristic C is the 
class of languages L such that there is a L' G C whereby, for almost every n, the symmetric difference 
(L n {0, l} n )A(L' n {0, l} n ) has cardinality less than 2 n /n. 18 (That is, there is a language in C that 
"agrees" with L on at least a 1 — 1/n fraction of inputs.) The infinitely often version /o-Heuristic C is 
defined analogously. 

Theorem 15 NEXP / BPP if and only if for all e > 0, BPP C /o-HeuristicZPTIME[2 n£ ]/n e . 

This extends an amazing result of Impagliazzo and Wigderson [13] that EXP 7^ BPP if and only if for 
all e > 0, BPP C ;o-HeuristicTIME[2 nE ]. It is interesting that NEXP versus BPP, a problem concerning 
the power of nondeterminism, is equivalent to a statement about derandomization of BPP without nonde- 
terminism. Theorem 15 should also be contrasted with the NEXP vs P/poly equivalence of IKW [12]: 
NEXP <jt P/poly if and only if MA C /o-NTIME[2 n£ ]/n e , for all e > 0. 

18 N.B. This is a weaker definition than usually stated, but it will suffice for our purposes. 
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Proof of Theorem 15. First, assume BPP is not in /o-HeuristicZPTIME[2 nE ]/n e for some s. Then 
BPP % /o-ZPTIME[2™ e ]/n £ , so by Theorem 6 we have that RTIME[2°( n )] has size-?i c seeds, which im- 
plies REXP = EXP. But we also have BPP is not in /o-HeuristicTIME[2 n ], so by Impagliazzo and 
Wigderson [13] we have EXP = BPP. Therefore REXP = BPP. But this means NP C BPP, so by Ko's 
theorem [18] we have NP = RP. Finally, by padding, NEXP = REXP = BPP. 

For the other direction, suppose NEXP = BPP and BPP C zo-HeuristicZPTIME[2 n£ ]/n e for all e > 0. 
We wish to prove a contradiction. The two assumptions say that NEXP C /o-HeuristicNTIME[2 n£ ]/n £ 
for all e > 0. NEXP = BPP implies NEXP = EXP, and since NE has a linear-time complete language, 
we have NTIME[2°( n )] C TIME[2°( nC )] for some constant c. (More precisely, the SUCCINCTHALTING 
problem from Theorem 1 can be solved in 2°( nC ) time for some c, and every language in NTIME[2°( n )] can 
be reduced in linear time to SUCCINCTHALTING.) As a consequence, 

EXP = NEXP C p| /o-HeuristicNTIME[2 n£ ]/ri £ C f] /o-HeuristicTIME[2° (nC) ]/™ e . 

e>0 e>0 

The last inclusion can be proved as follows: let L £ P| e >o 'oHeuristicNTIME[2" e ]/n e be arbitrary, and let 
L' € f| £ >o NTIME[2 n£ ]/n £ be such that (L n {0, 1}")A(L' n {0, l} n ) < 2 n /n on infinitely many n. This 
means that, for any e, V can be solved using a collection of nondeterministic machines {M n } running in 
2 n " time such that M n solves all instances on n bits and the description of M n can be encoded in 0{n £ ) bits. 
To get a collection of equivalent deterministic machines, let M n be the advice for inputs of length n; on any 
input x of length n, call the 2°^ time algorithm for SUCCINCTHALTING on the input (M n ,x, 6(2™ E )), 
where b(m) is the binary encoding of m. Using standard encodings, this instance has n + 0{n e ) length, 
hence it is solved deterministically in 2°^ n > time. 

However, the above inclusion is false, by direct diagonalization. That is, we can find an L £ EXP such 
that L £ /o-HeuristicTIME[2°( nC )]/n 1 / 2 . Let {Mi} be a list of all 2 n ° time machines. We will give a 
2™ c+1 -time M diagonalizing (even heuristically) against all {Mi} with n 1 / 2 advice. For every n, M divides 
up its n-bit inputs into blocks of length B = 1 + n 1 / 2 + logn, with 2 n /B blocks in total. On input x of 
length n, M identifies the block containing x, letting x\, . . . , ig be the strings in the that block. Let {aj} 
be the set of all possible advice strings of length n 1 / 2 . The following loop is performed: 

1 /2 

Let So = {(j, k) | j = 1, . . . ,n, k = 1, . . . , 2 n }. For i = 1,...,B, decide that M accepts Xi iff 
the majority of Mj(xi, a^) reject over all (j, k) E Set Si to be the subset of Si-i containing those 

(Mj, afe) which agree with M on Xi. If = x then output the decision. 

Observe that M runs in B ■ n ■ 2°( nC ) < 0(2 nC+1 ) time. For every block and every i, we have \Si\ < 

1/2 

\Si-i |/2. Since \Sq \ = 2 n -n, this implies that \Sb \ = 0. So for every block, every pair (Mj , at) disagrees 
with M on at least one input. Therefore every pair (Mj, a^) disagrees with M on at least 2 n /B > 2 n /n 
inputs, one from each block, and this happens for almost all input lengths n. Summing up, for almost every 
n we have that M disagrees with every Mj and its n 1 / 2 bits of advice, on greater than a 1/n fraction of n-bit 
inputs. That is, L(M) € EXP but L(M) g" /o-HeuristicTIMEp ^]/?! 1 / 2 . □ 

5 Conclusion 

One of Ketan Mulmuley's recent prophesies is that "P ^ NP because P is big, not because P is small" [21]. 
That is to say, the power of efficient computation is the true reason we can prove lower bounds. The 
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equivalence in Theorem 1 between NEXP lower bounds and P-time useful properties can be viewed as one 
rigorous formalization of this intuition. We conclude with some open questions of interest. 

• Do NEXP problems have witnesses that are average-case hard for ACC? More precisely, are there 
NEXP predicates with the property that, for almost all valid witnesses of length 2°( n \ their corresponding 
Boolean functions on 0(n) variables are such that that no ACC circuit of polynomial size agrees with 
these functions on 1/2 + l/poly(n) of the inputs? Such predicates could be used to yield unconditional 
derandomized simulations of ACC circuits (using nondeterminism). The primary technical impediment 
seems to be that we do not think ACC can compute the Majority function, which appears to be necessary for 
hardness amplification (see [26]). But this should make it easier to prove lower bounds against ACC, not 
harder! 

• Equivalences for non-uniform natural properties? In this paper, we have mainly studied natural prop- 
erties decidable by uniform algorithms; however, the more general notion of P /poly -natural proofs has 
also been considered. Are there reasonable equivalences that can be derived between the existence of such 
properties, and lower bounds? 

• What algorithms follow from stronger lower bound assumptions? There is an interesting tension be- 
tween the assumptions "NEXP (jL P/poly" and "integer factorization is not in subexponential time." The 
first asserts nontrivial efficient algorithms for recognizing some hard Boolean functions (as seen in this 
paper); the second denies efficient algorithms for recognizing a non-negligible fraction of hard Boolean 
functions [16, 3]. An equivalence involving NP <f_ P/poly could yield more powerful algorithms for recog- 
nizing hardness. 
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A Appendix: RP-natural implies P-natural 



Here we show how one can generically "derandomize" natural properties. The formal claim is: 

Theorem 16 If there exists a RP-natural property P useful against a class C, then there exists a P-natural 
property P' against C. 

That is, suppose there is a randomized algorithm that can distinguish hard functions from easy functions 
with one-sided error — the algorithm may err on some hard functions, but never on any easy functions. Then 
we can obtain a deterministic algorithm with essentially the same functionality. The idea behind P' is 
directly inspired by other proofs in this paper: we split the input string T into small substrings, and feed the 
substrings as inputs to P while the whole input string T is used as randomness to P. 

Suppose A is a randomized polytime algorithm taking n bits of input and n k ~ 2 bits of randomness (for 
some k > 3), deciding a large and useful property against n c -size circuits for every c. For concreteness, let 
us say that A accepts some l/n fe -fraction of n-bit inputs with probability at least 2/3, and rejects all n-bit 
truth tables of (log n) c -size circuits, where b > k (making b larger is only a weaker guarantee). Standard 
amplification techniques show that, by increasing the randomness from n k ~ 2 to n k , we can boost the success 
probability of A to greater than 1 — l/4 n . 

Our deterministic algorithm A' will, on n-bit input T, partition T into substrings T\,... ,T n i-i/ k of 
length at most n l / k each, and accept if and only if A(Ti, T) accepts for some i. 

First, we show that A 1 satisfies largeness. Consider the set R of n-bit strings T such that for all n 1 ' k - 
bit strings x, A(x, T) accepts if and only if A(x, T') accepts for some n-bit T'. As there are only 2 nl/k 
strings on n 1 ' fe bits, and the probability that a random ?7,-bit T works for a given n 1 / fc -bit string is at least 
1 _ \/4 nl/k , we have (by a union bound) that \R\ > 2 n • (1 - 2 n ' /k /4 nl/ *) > 2 n • (1 - l/2 nlA ). 

Now consider the set S of all n-bit strings T = T\ ■ ■ ■ T i-i/k (where for all i, |Tj| = n 1/,fc ) such that 
A(Ti, T') accepts for some i and some n-bit T' . Since there are at least t = 2" 1/fc /n b ^ k such strings Ti of 
length n lj/k (by largeness of A), the cardinality of S is at least 



/ , \ 7j l — 1/fe 1 r,n X l h , , . , . „l-l/fc 1 

n 1 -^ . t . 2 - 1/k -t) = n 1 "^ . 2 . . A _ l/nb/k \ n 



as this expression just counts the number of strings T with exactly one Tj from the t strings accepted by 
A. Since b > k, (1 - i/n^)™ 1 " 1 /*-! > i/ e> an d the above expression simplifies to 0(2 n /n l l k ~ l+b / k ). 
Therefore, there is a constant e = b/k + 1/k — 1 such that \S\ > Q,(2 n /n e ). 

Observe that, if T G S n R, then A(Ti,T) accepts for some i (where Ti is defined as above). Applying 
the inequality \S n R\ > \S\ + \R\ - 2 n , there are at least 2 n (l/n e - l/2 nl/fc ) strings such that A(T h T) 
accepts for some i. This is at least 2 n /n e+l for sufficiently lai - ge n, so A' satisfies largeness. 

Second, we show that A' is useful. Suppose for a contradiction that A'(T) accepts for some T with 
(log \T\) C size circuits, where c is an arbitrarily large (but fixed) constant. Then A(Ti,T) must accept for 
some i. Because A is useful against n rf -size circuits for all d, it must be that Tj cannot have (log |Tj|) c+1 size 
circuits. However, recall that if a string T has (log |T|) C size circuits, then by Lemma 1, every |T| -^-length 
substring T» of T has circuit complexity at most (log \T\) C + (log \T\) 1+ °^ < 2 • (k ■ log |Tj|) c . As k is a 
fixed constant, this quantity is less than (log |Tj|) c+1 when |Tj| is sufficiently large, a contradiction. 
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